Refchecker Logic Functionality #14213

ravatex · 2025-10-31T14:21:30Z

This PR includes all of the logic components of the RefCheck functionality.

With this we can implement refcheck in the CLI and GUI of jabref.

The added class, RefChecker, searches through different Fetchers to find matching or similar entries and returns either Fake, Real, or Unsure with related found entries.

To achieve this a new method in the DuplicateCheck class has been made, that finds the similarity of different Entries and returns a double between 0 and 1 depending on how similar the entries are.

Steps to test

Read the Unit Tests and run them to see how the classes are meant to work.

Mandatory checks

I own the copyright of the code submitted and I license it under the MIT license
I manually tested my changes in running JabRef (always required)
I added JUnit tests for changes (if applicable)
[/] I added screenshots in the PR description (if change is visible to the user)
[/] I described the change in CHANGELOG.md in a way that is understandable for the average user (if change is visible to the user)
[/] I checked the user documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request updating file(s) in https://github.com/JabRef/user-documentation/tree/main/en.

…lass

…cate check

jablib/src/main/java/org/jabref/logic/database/DuplicateCheck.java

…ity class

ravatex · 2025-11-01T18:33:40Z

In my implementation of DuplicateCheck, should I ignore any non-standard fields?

Would that be preferable?

subhramit

Hi, good work so far. I am leaving some comments.

subhramit · 2025-11-01T18:28:46Z

jablib/src/main/java/org/jabref/logic/database/DuplicateCheck.java

+     * @return number [0,1] 1 representing the same (one potentially having more fields), 0 representing completely different
+     */
+    public double degreeOfSimilarity(final BibEntry one, final BibEntry two) {
+        return one.getFields((f) -> two.getField(f).isPresent())


Avoid using abbreviations

subhramit · 2025-11-01T18:29:08Z

jablib/src/main/java/org/jabref/logic/database/DuplicateCheck.java

+                  .stream().mapToDouble((field) -> {
+                    String first = one.getField(field).get();
+                    String second = two.getField(field).get();
+                    return new StringSimilarity().similarity(first, second);


Don't create a new StringSimilarity instance for each field

subhramit · 2025-11-01T18:30:14Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+    public RefChecker(
+            DoiFetcher doiFetcher,
+            ArXivFetcher arXivFetcher) {


Only two parameters, so can be in a single line. Use one in each line when they are >=4

subhramit · 2025-11-01T18:34:14Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+    public RefChecker(
+            DoiFetcher doiFetcher,
+            ArXivFetcher arXivFetcher) {
+        this(doiFetcher, arXivFetcher, new CrossRef(), new DuplicateCheck(new BibEntryTypesManager()));


Try to inject bibentrytypesmanager via constructor. If you are unable to, use the injector (Injector.instantiateModelOrService(BibEntryTypesManager.class))

subhramit · 2025-11-01T18:35:36Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+        }
+
+        Optional<BibEntry> other = fetcher.performSearchById(doi.get().asString());
+        return other.map(o -> compareReferences(entry, o))


Avoid abbreviations

subhramit · 2025-11-01T18:50:17Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+    public static final class Fake extends ReferenceValidity {
+        public boolean equals(Object o) {
+            return o.getClass() == Fake.class;


(Note: Don't take this suggestion right away, let another maintainer confirm)
Creating a new Fake every time seems a bit odd as unlike Real, it is a stateless marker. This should be a singleton.
Maybe do something like

public static final Fake INSTANCE = new Fake(); private Fake() {} @Override public boolean equals(Object o) { return o instanceof Fake; }

and for the hashcode return an arbitrary constant.
This way, you can just do something like return Fake.INSTANCE instead of return new Fake().

@calixtus do you have a better idea? Something feels odd about the design.

subhramit · 2025-11-01T18:54:42Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+            Set<BibEntry> matchingReferences = new HashSet<>();
+            matchingReferences.add(matchingReference);
+            this.matchingReferences = matchingReferences;


Suggested change

Set<BibEntry> matchingReferences = new HashSet<>();

matchingReferences.add(matchingReference);

this.matchingReferences = matchingReferences;

this.matchingReferences = new HashSet<>(Set.of(matchingReference));

subhramit · 2025-11-01T19:16:15Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+                return this;
+            }
+            if (other instanceof Unsure otherUnsure && this instanceof Unsure thisUnsure) {
+                otherUnsure.addAll(thisUnsure);


An or method, in principle, should return a new result and not mutate its inputs

subhramit · 2025-11-01T19:18:12Z

jablib/src/test/java/org/jabref/logic/integrity/RefCheckerTest.java

+    @Test
+    void validateListOfEntriesTest() throws FetcherException {
+        List<BibEntry> entries = List.of(realEntry, realEntryNoDoi, fakeEntry);
+        var e = refChecker.validateListOfEntries(entries);


Remove abbreviations and var usages from this file.
(usually we don't fret so much about tests but in case we ever need to debug them, readability always helps)

subhramit · 2025-11-01T19:20:37Z

jablib/src/test/java/org/jabref/logic/integrity/RefCheckerTest.java

+    @Test
+    void closeToRealEntry() throws FetcherException {
+        RefChecker.ReferenceValidity rv = refChecker.referenceValidityOfEntry(closeToRealEntry);
+        System.out.println(((RefChecker.Unsure) rv).matchingReferences);


… modifying the old one, small code quality changes

ravatex · 2025-11-02T15:01:00Z

Should I ignore internal fields for degreeOfSimilarity, they are not fields that directly relate to the reference/bib so maybe they should be ignored? That makes sense at least from the RefChecker standpoint, but I could just make another method, that ignores internal fields and another that doesn't.

koppor · 2025-11-03T07:16:08Z

This is a difficult question.

Use case 1: Papers of a PDF and a fetcher: These fields probably won't appear.

Use case 2: Paper by fetcher is added to library: JabRef fields can be ignored.

Use case 3: Paper of library A is copied to library B: As user, I want the two entries being identified as similar even if "my" grouping, attached files, etc are different.

Result: Yes, can be ignored. Has to be documented at JavaDoc.

calixtus · 2025-11-03T07:18:27Z

They can be ignored, a fetcher will never return a special field value, they are JabRef specific.

koppor · 2025-11-03T13:39:58Z

jablib/src/test/java/org/jabref/logic/database/DuplicateCheckTest.java

+    @Test
+    void degreeOfSimilarityOfSameEntryIsOne() {
+        assertEquals(1.0, duplicateChecker.degreeOfSimilarity(getSimpleArticle(), getSimpleArticle()));
+        assertEquals(1.0, duplicateChecker.degreeOfSimilarity(getSimpleInCollection(), getSimpleInCollection()));
+    }


Please only a single assertion per method - see https://devdocs.jabref.org/code-howtos/testing.html

I think, you can rewrite using @ParamterizedTest.

I think, this can also be merged entriesWithNoMatchingFieldHaveNoSimilarity and more.

koppor · 2025-11-03T13:41:47Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+
+    public static abstract sealed class ReferenceValidity permits Real, Unsure, Fake {
+
+        public ReferenceValidity or(ReferenceValidity other) {


Please add JavaDoc (///) for this.

koppor · 2025-11-03T13:43:48Z

jablib/src/test/java/org/jabref/logic/integrity/RefCheckerTest.java

+            assertEquals(t1, t1.or(t2));
+            assertEquals(t1, t1.or(t3));
+            assertEquals(t2, t3.or(t2));


I don't get this... t1 is always equal to t1. Why is t1.or(t2) wirtten down there? 😅

koppor · 2025-11-03T13:44:23Z

jablib/src/test/java/org/jabref/logic/integrity/RefCheckerTest.java

+    }
+
+    @Test
+    void findsRealEntry() throws FetcherException {


I think, all tests can be rewritten to @ParamterizedTest

koppor · 2025-11-03T13:45:31Z

jablib/src/test/java/org/jabref/logic/integrity/RefCheckerTest.java

+                        StandardField.PDF,
+                        StandardField.PS,
+                        StandardField.URL,
+                        StandardField.DOI,
+                        StandardField.FILE,
+                        StandardField.ISBN,
+                        StandardField.ISSN)));


Not sure why we need all of them - PDF and PS are outdated.

koppor · 2025-11-03T13:46:32Z

jablib/src/main/java/org/jabref/logic/database/DuplicateCheck.java

+     * If the result is one, it means that there was at least one common field
+     * and all the common fields were the same.
+     * <p>
+     * Similar entries have a score of above 0.8


0.8 should be constant in DuplicateCheck / linked from there.

koppor · 2025-11-03T13:47:16Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+    DoiFetcher doiFetcher;
+    ArXivFetcher arxivFetcher;
+    CrossRef crossRef;
+    DuplicateCheck duplicateCheck;


These should be private?

koppor · 2025-11-03T13:48:14Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+    /**
+     * Tries to find the best reference validity
+     * among current ways. If any of the methods signal
+     * that it is real, it returns early.
+     * <p>
+     * DoiFetcher -> CrossRef -> ArxivFetcher
+     *
+     * @param entry entry checking
+     * @return the reference validity
+     * @throws FetcherException any error from fetchers
+     */


Try to use Markdown as JavaDoc format. Otherwise you need to escape >.

koppor · 2025-11-03T14:16:56Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+        public int hashCode() {
+            return Objects.hashCode(Fake.class);
+        }


No neeed for an implementation, JDK's default works well in most cases

Override is only necessary if equals is overridden: https://thefinestartist.com/effective-java/09

koppor · 2025-11-03T14:22:30Z

jablib/src/main/java/org/jabref/logic/integrity/RefChecker.java

+        ReferenceValidity get() throws FetcherException;
+    }
+
+    public static abstract sealed class ReferenceValidity permits Real, Unsure, Fake {


Maybe

record(Validity validtiy, @Nullable BibTeX otherEntry)

And Valildity being an enum - and otherEntry allowed for null in the fake entry.

I am not sure if there could be a group of other entries in some cases; but this design would be more future proof and more readable. @subhramit

ravatex added 15 commits October 26, 2025 20:51

add refchecker class which does the core functionality of refchecking

50c3ca9

add made some methods public that are useful

eeec045

removed unused Parser argument

735f985

add tests for refchecker and modified refchecker to add equals methods

be4d588

added validation for entry lists and parsing from input

9939899

added test for list of entries

037d243

removed parse and validate, there is no reason to have that in this c…

9725af0

…lass

added documentation

43dc552

added degree of similarity to see if entries have similar fields

b9027da

added tests for degree of similarity

ead14fd

changed refchecker to work with degree of similarity instead of dupli…

e442b24

…cate check

added tests for updated refchecker with unsure

151aaa7

improved degree of similarity docs

c8bfb21

fixed checkstyle

6d783e9

fix rewrite run

f77341a

Siedlerchr reviewed Nov 1, 2025

View reviewed changes

jablib/src/main/java/org/jabref/logic/database/DuplicateCheck.java Show resolved Hide resolved

refactored duplicateCheck#degreeOfSimilarity to use the StringSimilar…

a33eb13

…ity class

subhramit requested changes Nov 1, 2025

View reviewed changes

subhramit requested a review from koppor November 1, 2025 19:28

ravatex added 4 commits November 2, 2025 00:40

fix small changes for code quality

6c1aa0e

changed the or functionality to return a new Unsure object instead of…

80b8058

… modifying the old one, small code quality changes

changed var to qualified names and removed a print statements

6f0e9c7

fixed degree of similarity documentation

b98b78a

koppor requested changes Nov 3, 2025

View reviewed changes


		public static abstract sealed class ReferenceValidity permits Real, Unsure, Fake {

		public ReferenceValidity or(ReferenceValidity other) {

Uh oh!

Refchecker Logic Functionality #14213

Are you sure you want to change the base?

Refchecker Logic Functionality #14213

Conversation

ravatex commented Oct 31, 2025

Steps to test

Mandatory checks

Uh oh!

Uh oh!

ravatex commented Nov 1, 2025

Uh oh!

subhramit left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

subhramit Nov 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ravatex commented Nov 2, 2025

Uh oh!

koppor commented Nov 3, 2025

Uh oh!

calixtus commented Nov 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

subhramit Nov 1, 2025 •

edited

Loading